Image or video compression artifacts are introduced by digital image or video compression algorithms as a result of inappropriate distribution of available bits. Block artifacts, which are one of the most annoying compression artifacts, are perceived as unwanted discontinuities at the block boundaries of compressed pictures. Block artifacts become particularly more visible with high definition formats and high resolution displays. The manual determination by an operator of the locations of artifacts as part of a re-encoding process is not feasible in many image and video coding applications as a process can be expensive because of the amount of time required for finding such block artifacts. Therefore, it is necessary to perform automatic block artifact detection and reduce the blocky appearance in compressed pictures.
Once block artifacts are detected in a particular picture, post-processing algorithms or re-encoding with new encoding parameters can be applied to the picture in order to correct the blocky appearance of the picture. The post-processing or re-encoding algorithms can make fine adjustments to encoding parameters based on detected block artifact locations in order to get better picture quality. The strength of a block artifact is another important aspect of artifact detection which can guide the post-processing or re-encoding algorithms to fine tune their parameters to further improve the picture quality.
In some applications which perform automatic artifact correction in compressed video sequences with limited bit-rate constraints, the optional step can be performed of having an overall blockiness artifact metric per picture in order for the video encoder to decide which pictures to re-encode with the available bits. That is, some encoded pictures will have more block artifacts than other encoded pictures. Hence, the blockiness artifact metric is a standard way of measuring the blockiness of respective encoded pictures against each other.
Consequently, there is a need for algorithms that automatically detect block artifacts and determine the strength of the artifact per block and per picture.
Various prior art approaches have been proposed for detecting block artifact and for correcting such artifacts in low to high bit rate images using various video correction techniques. Few of these prior art approaches however only focus on artifact detection and metric determination. Furthermore, the prior art approaches appear to not address the problem of block artifact detection for pictures or video compression methods that make use of respectively high bit-rates.
In a first prior art approach, a block artifact metric determination algorithm is proposed. The first prior art approach compares the mean of the pixel values in a macroblock with the mean of its neighboring macroblocks. The individual differences are summed to obtain an overall blockiness metric. A simple edge detector is applied to exclude real edges from artifacts. A second prior art approach proposes an artifact detection metric that uses pixel values at the block boundaries. The two consecutive pixel values in two sides of the block boundary are filtered and the filtered value is summed to obtain an overall blockiness metric. In a third prior art approach, a block artifact metric is proposed which works on the luminance values of the images. The third prior art approach is similar to the first prior art approach in the sense that it computes and compares the mean values of blocks, with the exception that only those blocks with constant intensity are considered.
The first, second, and third prior art approaches do not consider perceptual factors and other properties of compressed images and video and, therefore, are not directly applicable to high resolution displays. The first, second, and third prior art approaches are mainly tested with video or image content compressed with low-to-medium bit rates. High definition video compression requires higher quality and higher bit rates. Even the smallest artifacts which are not visible in smaller displays such as cell phones or personal computer monitors are visible in high resolution displays.
A fourth and a fifth prior art approach consider perceptual factors. In the fourth prior art approach, a block metric is derived by considering the sum of the pixel differences at the block boundaries, where the differences are weighted by the pixel intensities. Gray values between 70 and 90 (in an 8 bit gray scale image, for example) are weighted more than dark and bright pixels. In the fifth prior art approach, a block artifact detection scheme is proposed that considers some perceptual factors. In particular, the background luminance and background activity are used as perceptual features which affect the visibility of the artifacts. In order to derive the perceptual model parameters, visual experiments are performed. Images including edges with different lengths and amplitudes with different backgrounds are shown to human subjects and the visibility of the edge patterns are determined based upon the visual perception of such subjects. Based on the subjective results, perceptual parameters that mask the artifacts are found. Some of the perceptual features used in the fifth prior art approach are useful such as background activity, and some of them are interesting but highly complex such as a frequency domain processing to obtain the perceptual parameters that incorporated into a video encoding process.
An in-loop de-blocking filter is another tool for suppressing block artifacts in video compression applications based on the International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) Moving Picture Experts Group-4 (MPEG-4) Part 10 Advanced Video Coding (AVC) standard/International Telecommunication Union, Telecommunication Sector (ITU-T) H.264 recommendation (hereinafter the “MPEG-4 AVC standard”). The in-loop de-blocking filter is proposed both for artifact detection and correction. The filtering is adaptive to the blockiness strength which is determined by compressed domain information. The filtering smoothes over the reconstructed pixels near the block boundaries. However in very-high bit rate encoding of high-resolution content, the spatial details which we would like to preserve are lost or blurred when the de-blocking filter is turned on.
Thus, in the prior art, the problem of detecting block artifacts is not completely addressed and/or considered with respect to the properties of high definition content and various corresponding perceptual features.